Remote interaction with content of a transparent display

ABSTRACT

A computer to receives a first set of data associated with movements of a second hand of the user on a first side of the first hand of the user and receiving a second set of data associated with movements of the second hand of the user on a second side of the first hand of the user. The first set of data is mapped to a first side of the transparent display and the second set of data is mapped to a second side of the transparent display. The approach includes receiving a first set of sensor data associated with movements of the second hand of the user on the first hand of the user and determining a manipulation of a content on a side of the transparent display corresponding to the first set of sensor data and sending instructions to the transparent display to display the content.

BACKGROUND OF THE INVENTION

The present invention relates generally to the field of computer displaytechnology and more particularly to remotely providing input associatedwith content on a transparent display.

A transparent display is an electronic display that allows a user toview content shown on either side of a display screen while still beingable to see to the real world through the transparent display. Emergingcommercial uses of transparent displays, for example, include augmentedreality applications, virtual reality gaming, and the use of transparentdisplays in commercial settings such as a store window providingpromotions, messages on current new products, or additional visualsoverlaying store window displays.

Typically, transparent displays may add, erase, move, or otherwisemanipulate display content such as images, words, graphs, or otherdigitally generated content in response to a user input provided througha user's touch on the transparent display or by a user gesture capturedby a digital camera. In some cases, transparent displays may add,delete, or otherwise manipulate displayed content in response to a userinput on a computing device such as a smart phone, tablet, computer, orthe like using a conventional user interface such as a touch screen,keyboard, or mouse of the computing device.

SUMMARY

Embodiments of the present invention disclose a method, a computerprogram product, and a system for one or more computers to receive afirst set of mapping sensor data associated with a plurality ofsystematic movements of the second hand of the user on a first side ofthe first hand of the user and map the first set of mapping sensor datato the first side of the first hand of the user. The method includes oneor more computer processors mapping the first side of the first hand ofthe user to the first side of the transparent display. Furthermore, themethod includes one or more computer processors receiving a second setof mapping sensor data associated with a plurality of movements of thesecond hand of the user on a second side of the first hand of the userand mapping the second set of mapping sensor data to the second side ofthe first hand of the user. Furthermore, the method includes one or morecomputer processors mapping the second side of the first hand of theuser to the second side of the transparent display. The method includesone or more computer processors receiving a first set of sensor dataassociated with one or more movements of the second hand of userassociated to the first hand of the user, wherein the one or more setsof mapping sensor data and the first set of sensor data are determinedbased on muscle movements or vibrations detected from the plurality ofsystematic movements of the second hand of the user on the first hand ofthe user and the one or more movements of the second hand of the user onthe first hand of the user, and wherein the one or more sets of sensordata and the first set of sensor data are received from one or moresensors in at least one of: a smart ring, a smart watch, a fitnesstracking device, a smart armband, or a sensor glove with processingcapability associated with the second hand. Additionally, the methodincludes one or more computer processors determining the first set ofsensor data corresponds to at least one content on the transparentdisplay and determining whether the first set of sensor data isassociated with more than one side of the transparent display.Responsive to determining the first set of sensor data is not associatedwith more than one side of the transparent display, the method includesone or more computer processors determining a manipulation of the atleast one content on a side of the transparent display corresponding tothe first set of sensor data and sending one or more instructions to thetransparent display to display the at least one content based, at leastin part, on the determined manipulation of the at least one content andthe mapping of the transparent display. Responsive to determining thefirst set of sensor data is associated with more than one side of thetransparent display, the method includes one or more computer processorsdetermining that the at least one content is moving from one side of thetransparent display to a second side of the transparent display based,at least in part, on the first set of sensor data and determining anorientation of the at least one content on the second side of thetransparent display. Furthermore, the method includes one or moreprocessors sending one or more instructions to the transparent displayto display the at least one content on the second side of thetransparent display based, at least in part, on the first set of sensordata.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram illustrating a distributed dataprocessing environment, in accordance with at least one embodiment ofthe present invention.

FIGS. 2A and 2B are an illustration of a method to select and manipulatecontent on a side of a transparent display, in accordance with anembodiment of the present invention.

FIGS. 3A and 3B are an illustration of a method to transfer content froma side of a transparent display to another side of the transparentdisplay, in accordance with an embodiment of the present invention.

FIGS. 4A and 4B are an illustration of a method of manipulating data onboth sides of a transparent display, in accordance with an embodiment ofthe present invention.

FIGS. 5A and 5B are a flowchart depicting operational steps of a methodto manipulate content on a transparent display, in accordance with anembodiment of the present invention.

FIG. 6 is a block diagram depicting components of a computer system inaccordance with at least one embodiment of the present invention.

DETAILED DESCRIPTION

Embodiments of the present invention recognize that in some instances atransparent display may be too large to allow a user to provide an inputby touch on a second side of the transparent display without walkingaround the transparent display. Embodiments of the present inventionrecognize that a large transparent display can prohibit a user fromproviding input by touch simultaneously to both sides of the transparentdisplay. Embodiments of the present invention recognize that not allusers have access to imaging devices with associated image analysisprograms. Embodiments of the present invention recognize that not allusers desire a visual image analysis of arm or hand gestures captured bya digital imaging device or camera as a method to provide user input forthe manipulation of data or other content on a transparent display dueto potential visual image background noise from movements of othernearby individuals.

Embodiments of the present invention provide a method for themanipulation of content on both sides of a transparent display.Embodiments of the present invention allow a user to provideinstructions to a second side of a transparent display without reachingor walking around the transparent display. Embodiments of the presentinvention provide a method for providing user input for remoteinteractions of a user to a transparent display. Embodiments of thepresent invention include a method for receiving user input using sensordata associated with a user's muscle movements or vibrations created asa user's first-hand contacts or moves on the user's second hand and thereceived sensor data is correlated to a content on a transparent displayor an action associated with the transparent display. Embodiments of thepresent invention provide a method to receive user inputs such as sensordata associated with a content on a transparent display or an action ofa transparent display without erroneously receiving instructionsgenerated by another nearby user moving his or her hands or arms.

FIG. 1 is a functional block diagram illustrating a distributed dataprocessing environment, generally designated 100, in accordance with atleast one embodiment of the present invention. FIG. 1 provides only anillustration of one implementation and does not imply any limitationswith regard to the environments in which different embodiments may beimplemented. Many modifications to the depicted environment may be madeby those skilled in the art without departing from the scope of theinvention as recited by the claims.

As depicted, distributed data processing environment 100 includescomputer 120, transparent display 140, and sensors 150 allinterconnected over network 110. Network 110 can include, for example,wireless local area network (WLAN), a telecommunications network, alocal area network (LAN), a virtual LAN (VLAN), a wide area network(WAN), such as the Internet, or a combination of the these, and caninclude wired or wireless connections. Network 110 can include one ormore wireless networks and/or wired networks that are capable ofreceiving and transmitting data including wireless data such as sensordata, radio waves, infrared signals, electrical signals, and othersimilar signals for data transmission. In general, network 110 can beany combination of connections and protocols that will supportcommunications between computer 120, transparent display 140, andsensors 150 along with other computing devices (not shown) withindistributed data processing environment 100.

Computer 120 can be a smart watch, a smart ring, a smart armband, afitness tracking device, a smart phone, a laptop computer, a tabletcomputer, a desktop computer, or any programmable electronic devicecapable of communicating with various components and devices withindistributed data processing environment 100, via network 110. Ingeneral, computer 120 represents any programmable electronic device orcombination of programmable electronic devices capable of executingmachine-readable program instructions and communicating with othercomputing devices within distributed data processing environment 100 viaa network, such as network 110. In various embodiments, computer 120includes sensors 124 capable of detecting a motion, a movement, and/orlocation of a first hand of a user or a digit of the first hand of theuser with respect to a second hand of the user. In some embodiments,computer 120 includes an instance of user interface (UI) 125. In variousembodiments, computer 120 receives sensor data from one or both ofsensors 124 and sensors 150. Computer 120 accesses transparent display140 and other computer devices (not depicted in FIG. 1) via network 110,for example, to receive sensor data and provide inputs to transparentdisplay 140 for manipulation and display of content on transparentdisplay 140. Computer 120 may include internal and external hardwarecomponents, as depicted and described in further detail with respect toFIG. 6.

Control program 122 resides in computer 120. In various embodiments,control program 122 receives sensor data from sensors 124 and/or sensors150. In various embodiments, control program 122 determines an action ora manipulation of selected content on transparent display 140 based, atleast in part, on received sensor data (e.g., as user inputs). In anembodiment, control program 122 receives user input for a selection ofcontent for manipulation of content on transparent display 140 from oneor more user selected buttons on UI 125 (e.g., when UI 125 is atwo-sided remote input device). In one embodiment, control program 122receives user input for a selection of content for manipulation ofcontent on transparent display 140 from a touch on a two-sided touchscreen used for UI 125.

In various embodiments, control program 122 maps a hand of a user totransparent display 140. For example, control program 122 maps a righthand of a user to transparent display 140 based, at least in part, onreceived sensor data associated with a movement of a digit such as afinger or a thumb of a left hand of the user along or on the user'sright hand. In various embodiments, control program 122 maps one or moresides of a user's hand to one or more sides of transparent display 140.For example, control program 122 moves a finger of the left handsystematically around the palm of the user's right hand to map the palmof the right hand to a front side of transparent display 140. In oneembodiment, control program 122 maps a grid of buttons on each side of atwo-sided remote input device design for UI 125 to transparent display140 (e.g., a first side of UI 125 to a first side of transparent display140). In an embodiment, control program 122 includes instructionsmapping UI 125 as a two-sided touch screen to transparent display 140.

After completion of mapping sensor data and/or UI 125 to transparentdisplay 140, in various embodiments, control program 122 receives, asuser input, sensor data from sensors 124 or sensors 150 associated witha movement or a touch of a first hand of the user on a second hand ofthe user. Control program 122 can receive sensor data associated withone or more sides of a user's mapped hand or associated with a gesture(e.g., a cupped palm or a first) of the user's mapped hand. In anembodiment, control program 122 receives a user input from UI 125 as atwo-sided remote input device or UI 125 as a two-sided touch screen.Control program 122 determines an action or manipulation of a selectedcontent on transparent display 140 corresponding to the received sensordata or UI 125 input. In various embodiments, control program 122determines an orientation for content transferred or moved from one sideof transparent display 140 to the other side of transparent display 140(e.g., reverses or flips content appropriately to display content in thesame manner on the second side of transparent display 140 as viewed onthe first side of transparent display 140). In various embodiments,control program 122 provides instructions to transparent display 140 toperform one or more of the following: manipulate content, selectcontent, move content, orient content, display content, delete content,add content, rotate content, or perform an action (e.g., lockingtransparent display) on one or both sides of transparent display 140 inresponse to received sensor data received in association with one orboth sides of a user's second hand. Control program 122 receives andsends data or instructions to and from transparent display 140. Invarious embodiments, control program 122 sends and receives user input(e.g., sensor data) from sensors 124, sensors 150, or UI 125.

Sensors 124 and sensors 150 capture and send data to control program 122generated by movements or gestures of a user's first hand on a user'ssecond hand. In various embodiments, sensors 124 and/or sensors 150provide sensor data to control program 122 as a user input to determinea selection of content on transparent display 140, a manipulation ofcontent on transparent display 140, or an action of transparent display140. In various embodiments, sensors 124 and/or sensors 150 are one ormore sensors capable of detecting muscle movements, vibrations, or othersimilar, non-visual indication of hand motions such as a movement ortouch of a user's first hand on one or more locations on either side ofa user's second hand. Sensors 124 and 150 can employ one or more ofknown motion and/or location determination methods such as vibrationsensing or muscle movement sensing devices associated with a first handof a user. For example, sensor 124 and sensors 150 determine musclemovements, vibrations associated with a movement of a hand along anotherhand, or a location on a hand of a user using one or more of knownsensing technologies such as precision vibration sensors, surfaceelectromyography sensors, gyroscope sensors, reflected infrared light,piezoelectric films, or the like. In some embodiments, sensors 124and/or sensors 150 determine a location of a hand using one or moreknown location determination methods (e.g., as used in roboticstechnology) such as sensor gloves or multiple sensors in a smartarmband, smart watch and the like. In various embodiments, computer 120includes sensors 124 (e.g., as sensors integrated in a smart watch, afitness tracking device, a smart ring, a smart armband, a sensor glove,a smart sensor glove, etc.). In an embodiment, computer 120 receivessensor data via network 110 from sensors 150. For example, sensors 150are not integrated in computer 120 but are independent sensors such as apiezoelectric film or vibration sensors attached directly to one or bothhands of the user. For example, sensors 150 can be attached to theuser's skin along a hand, a wrist, or an arm, in a glove, on afingernail, a ring, a wristband, or the like without processingcapability or limited processing capability. In various embodiments,sensors 124 or sensors 150 provide data to control program 122associated with a systematic movement of a user's first hand over eachsurface of a user's second hand used to map a surface of the user'ssecond hand to a side of transparent display 140.

User interface (UI) 125 provides an interface for a user of computer 120to control program 122, transparent display 140, and other computingdevices in distributed data processing environment 100 (not shown). Invarious embodiments, user interface 125 may be a two-sided remote inputdevice, a two-sided touch screen, a graphical user interface (GUI) or aweb user interface (WUI). In various embodiments, UI 125 can receiveuser input such as instructions for operation, and include theinformation (such as graphics, text, and the like) communicated tocontrol program 122 and transparent display 140 to present informationto a user on transparent display 140. In another embodiment, UI 125 mayalso be mobile application software that provides an interface betweenthe users of computer 120 and transparent display 140. User interface125 enables computer 120 to receive a user selection of content, alocation for content, or an action associated with transparent display140. In an embodiment, UI 125 is a two-sided remote input device with agrid of buttons on each side of the two-sided remote input device (e.g.,UI 125) each button mapped to a corresponding location on transparentdisplay 140 and a row buttons on each side of UI 125 associated with anaction to occur on transparent display 140. UI 125 as a two-sided remoteinput device receives user inputs via a selection of one or more buttonson one or both sides of UI 125 in order to provide instructions totransparent display 140. In one embodiment, UI 125 is a two-sided touchscreen.

Transparent display 140 is a transparent display capable of presentingcontent such as pictures, graphs, and other electronic data on any sideof the display (e.g., on any sided of a multi-sided transparentdisplay). Content for transparent display 140 may include but, is notlimited to, images, graphs, numbers, symbols, words, files, videos,code, or other digital data displayed on transparent display 140. Invarious embodiments, transparent display 140 presents content on eitherside of transparent display. Transparent display 140 allows a user tosee displayed content and the environment behind the screen (e.g., theuser sees through the display to observe the background or actionsoccurring behind transparent display 140). Transparent display 140receives data and instructions via network 110 from control program 122.Responsive to instructions received from control program 122,transparent display 140 performs actions such as unlocking the display,deleting/adding content, or manipulating content such as movingidentified content from one side of transparent display 140 to anidentified location on the other side of transparent display 140. Invarious embodiments, transparent display 140 receives instructions ordata from control program 122 determined in response to a user inputreceived by sensors 124 or sensors 150 generating sensor data as a digitof the user's first hand moves with respect to one or both sides of theuser's second hand. In an embodiment, transparent display 140 receivesinstructions or data from UI 125 that is one of a two-sided touch screenor a two-sided remote input device which maybe a separate remote inputdevice or integrated in computer 120 (as depicted).

FIGS. 2A and 2B are an illustration of a method to select and manipulatecontent 203 on side 210A of transparent display 140 using data fromsensors in computer 120 on hand 201, in accordance with an embodiment ofthe present invention. As depicted, FIG. 2A includes hand 201, computer120 (e.g., a smart ring with sensors 124 not depicted), hand 205, palm205A, transparent display 140, side 210A, side 210B, and content 203(e.g., a glass of water). FIG. 2A depicts hand 201 with computer 120(e.g., a smart ring with sensors) touching hand 205 on palm 205A in afirst location sliding to a second location on palm 205A occurring aftercomputer 120 receives a user input for a generation of instructions totransparent display 140. FIG. 2B is a corresponding illustration oftransparent display 140 moving content 203 from the right side of side210A to the left side of side 210A. The movement of content 203 ontransparent display 140 is determined by computer 120 using controlprogram 122 (depicted in FIG. 1) based, at least in part, on receivedsensor data from sensors (depicted in FIG. 1) in computer 120 (e.g., asmart ring). Computer 120 receives sensor data such as vibration datadetected by sensors in computer 120 on the finger of hand 201 as thefinger moves along palm 205A of hand 205. Prior to the finger of hand201 touching hand 205, computer 120 has mapped palm 205A of hand 205 toside 210A of transparent display 140 (e.g., from previously receivedsensor data as hand 201 systematically traverses each side of hand 205).Based, at least in part, on received sensor data from sensors incomputer 120 as the finger moves across palm 205A and mapping of palm205A to side 210A of transparent display 140, computer 120 sendsinstructions to transparent display 140 to move content 203 from thefirst location to a second location on transparent display 140.

FIGS. 3A and 3B are an illustration of a method to transfer content 303from side 210A of transparent display 140 to side 210B of transparentdisplay 140, in accordance with an embodiment of the present invention.As depicted, FIG. 3A includes hand 201 with computer 120 (e.g., depictedas a smart ring), hand 205, palm 205A, and backside 205B of hand 205.FIG. 3B includes transparent display 140 with side 210A (e.g., frontside) and side 210B (e.g., backside), content 303 on side 210A viewedfrom the front side of transparent display 140, and content 303 on side210B viewed from the front side of transparent display 140.

Before receiving sensor data associated with hand 201 movements depictedin FIG. 3A, computer 120 has already mapped each side of hand 205 to acorresponding side of transparent display 140 (e.g., palm 205A mapped toside 210A) based on received sensor data. Pointer finger on hand 201touches palm 205A and slides around the edge of palm 205A to a secondlocation on backside 205B of hand 205 and sensors (not depicted) incomputer 120 provide sensor data associated to the movement of thepointer finger on hand 205 to computer 120 (e.g., to control program 122not depicted). In response to receiving sensor data associated with themovement of the pointer finger on hand 201 around hand 205, computer 120determines that content 303 is moved from side 210A to a second locationon side 210B of transparent display 140 as depicted in FIG. 3B and sendsinstructions to transparent display 140 to move content 303.

FIGS. 4A and 4B are an illustration of a method of manipulating data onboth sides of transparent display 140, in accordance with an embodimentof the present invention. Using previously performed mapping of eachside of transparent display 140 to each side of hand 205 of a user,computer 120C, a smart watch with sensors (not depicted) detectingmuscle movements associated with the movements of the digits of hand201, receives sensor data on the user's muscle movements from thesensors in computer 120C associated with the movement of a pointerfinger of hand 201 on backside of hand 205 and the movement of a thumbof hand 201 on palm side 205A of hand 205. Sensor data from sensors (notdepicted) in computer 120C provide data on muscle movements in theuser's wrist, using known methods known to one skilled in the art,associated with a sliding motion upward by the thumb on palm 205A ofhand 205 to a location on palm 205A and a sliding motion downward by thepointer finger touching a location on backside of hand 205 to a secondlocation on the backside of hand 205. In response to receiving thesensor data on the sliding motions and associated locations (e.g., frommapping of hand 205), computer 120 sends instructions to transparentdisplay 140 to move content 411 on side 210B of transparent displaydownward and content 412 on front side 210A of transparent display 140upward as depicted on transparent display 140 in FIG. 4B.

FIGS. 5A and 5B are a flowchart 500 depicting operational steps ofcontrol program 122 to manipulate content on transparent display 140, inaccordance with an embodiment of the present invention. It should alsobe noted that, in some alternative implementations, the functions notedin the block may occur out of the order noted in the figures. Forexample, two blocks shown in succession may, in fact, be executedsubstantially concurrently, or the blocks may sometimes be executed inthe reverse order, depending upon the functionality involved.

Control program 122 maps a palm of a hand of a user to a front side oftransparent display 140 (504). In various embodiments, control program122 receives sensor data or a user input on UI 125 indicating aninitiation of mapping of a hand (e.g., hand 205 in FIGS. 2A, 3A, and 4A)to transparent display 140. For example, when control program 122receives a command from a user input such as a selection of “mapping” ona pull-down menu to initiate mapping of a hand to transparent display140. In another example, control program 122 receives sensor data (e.g.,a set of sensor mapping data) associated with muscle movements orvibrations detected by sensors 124 or 150 indicating a finger on theuser's other hand (e.g., hand 201 in FIGS. 2A, 3A, and 4A) rubs along oragainst the knuckles of fisted hand 205 to initiate of a mapping of hand205. In an embodiment, control program 122 runs continuously in thebackground analyzing received sensor movement to determine an initiationof the mapping of hand 205 (e.g., when sensor data indicates a finger orthumb moving across the knuckles of fisted hand 205). In one embodiment,control program 122 initiates interactions such as mapping andgenerating instructions to transparent display 140 when computer 120with sensors 124 is in proximity to transparent display 140. Forexample, control program 122 initiates analysis or mapping of sensordata when computer 120 and/or sensors 124 are within ten feet oftransparent display 140. Computer 120 proximity to transparent display140 may be determined using known location determination methods such asan indoor positioning system, Wi-Fi based positioning system, beacons,global positioning system, and the like. In some embodiments, controlprogram 122 initiates analysis of received sensor data for mapping oftransparent display 140 by a received user input such as a button on asmart arm band or a smart watch for computer 120, a user input orcommand provided from a user's smartphone, tablet, or notebook (notdepicted) sent to computer 120 for the initiation of control program122. In an embodiment, control program 122 receives sensor data fromsensors 124 and/or sensors 150 associated with hand 205.

In various embodiments, control program 122 receives sensor data fromsensors 124 present in computer 120 on or associated with a user'ssecond hand (e.g., hand 201) as hand 201 moves around the palm (e.g.,palm 205A in FIGS. 2A, 3A, 4A) of hand 205 to map hand 205. Computer 120can be a smart watch, a smart ring, a smart armband, or the like on hand201 or on a user's finger, thumb, wrist or arm associated with hand 201of a user. In various embodiments, control program 122 receives sensordata associated with movements of hand 201 such as vibrations, musclemovements, motion data, infrared (IR) light changes, or other sensordata associated with hand 201 or muscle movements of hand 201 as hand201 (e.g., a digit of hand 201) traverses palm 205A. The received sensordata from sensors 124 in computer 120 on hand 201 provide sensor dataassociated with palm 205A of hand 205 including, for example, musclemovements or vibrations related to a topology and/or a location orlocations on palm 205A as a finger of thumb of hand 201 traverses palm205A. In various embodiments, control program 122 maps transparentdisplay 140 to palm 205A according to sensor data detected as hand 201or a digit of hand 201 systematically moves along hand 205. For example,control program 122 maps palm 205A of hand 205 to side 210A oftransparent display 140 based, at least in part, on sensor data receivedfrom sensors 124 as hand 201 moves around palm 205A. In an embodiment,sensors 150 provide sensor data associated with a movement detected onhand 201 along palm 205A of hand 205 used to map palm 205A to side 210Aof transparent display 140.

For example, control program 122 maps the sensor data associated withpalm 205A (e.g., sensor data indicating changes in topology of the palmof hand 205, muscle movement of hand 201, a muscle movement of a wristor an arm of hand 201, and/or locations of hand 201 with respect to hand205) to a grid representing palm 205A. In various embodiments, controlprogram 122 receives sensor data capturing vibrations or musclemovements (e.g., in hand 201) due to the topology of palm 205A orchanges a top surface of the user's palm 205A (e.g., hills due to fingerbones and muscles or valleys due to palm lines or creases) and usesreceived sensor data to map palm 205A of hand 205.

In one example, control program 122 receives sensor data (e.g., as userinput) from sensors 124 as a finger of hand 201 with a smart ring or asmart watch for computer 120 as the finger systematically moves aroundpalm 205A from vibrations occurring in hand 201 as it traverses hand205. Sensors 124 can include precision vibration sensing technologycapable detecting minor or small vibrations due to bumps or creases inpalm 205A. In various embodiments, control program 122 receives sensordata as a user's finger on hand 201 systematically moves around palm205A. For example, control program 122 receives sensor data generated bythe user's finger of hand 201 circling palm 205A in co-centric circlesfrom the outer edges of palm 205A to the center of palm 205A, byzigzagging fingers of hand 201 from right to left across the outer edgesof the palm, or by other similar motions of hand 201 to map palm 205A.

In various embodiments, control program 122 maps the grid for palm 205Ato a corresponding grid for transparent display 140. For example, aftercontrol program 122 receives sensor data from sensors 124 as one or morefingers of hand 201 trace or slide over palm 205A then, accordingly,control program 122 maps a grid associated with the topology of palm205A to a grid representing side 210A of transparent display 140. Invarious embodiments, control program 122 receives sensor data indicatinga completion of the mapping of a side of hand 205, for example, by threetaps of a finger or a thumb of hand 201 on a side of hand 205 aftercompleting the traverse of hand 201 over the side of hand 205 mapped.

Control program 122 maps a backside 205B of hand 205 of the user to side210B transparent display 140 (506). Using the methods discussed above instep 504, control program 122 receives sensor data from sensors 124(e.g., in a smart ring, in a smart armband, fitness tracking device,etc.) as hand 201 traverses the top surface of hand 205's backside(e.g., backside 205B of hand 205 opposite palm 205A). In variousembodiments, control program 122 maps the backside 205B of hand 205 toanother side (e.g., side 210B) of transparent display 140. For example,control program 122 maps a grid point associated with a location onbackside 205B of hand 205 to a corresponding grid point on side 210B oftransparent display 140. In various embodiments, control program 122maps each grid point associated with the topology of backside 205B ofhand 205 (e.g., based, at least in part, on received sensor data ofmuscle movement, vibrations, or the like of a finger of hand 201traversing backside 205B). In one embodiment, control program 122receives sensor data from sensors 150 for mapping sides of hand 205 toone or more sides of transparent display 140. In an embodiment, controlprogram 122 receives sensor data associated with one or both hands andone or more sides of a hand of the user for mapping hand 205 totransparent display 140. For example, sensors 150 can be a piezoelectricfilm on the surface of one or both sides of hand 205 providing sensordata on backside 205B for mapping to side 210B of transparent display140.

In an embodiment, control program 122 maps each side of a two-sidedremote input device (not depicted in FIG. 1) or a two-sided smart remoteinput device (e.g., UI 125 integrated in computer 120) to each side oftransparent display 140. In one embodiment, control program 122 mapseach side of a two-sided touch screen (e.g., UI 125) in computer 120 toeach side of transparent display 140. In some embodiments, controlprogram 122 maps a multi-sided remote input device to a multi-sidedtransparent display 140 (e.g., a rectangular remote input device for UI125 with input buttons on four sides to a four-sided transparent display140). In an embodiment, control program 122 maps multiple sides of hand205 to a multi-sided transparent display 140. For example, controlprogram 122 maps an area or circumference around palm 205A, backside205B, and the sides of hand 205 between them to a cylindricaltransparent display 140. In other example, control program 122 maps palm205A to one side of transparent display 140, backside 205B to a secondside of transparent display 140, and an outside area of hand 205 betweenpalm 205A and backside 205B on the little finger side of hand 205 to athird side of transparent display 140 (e.g., a three-sided transparentdisplay) based, at least in part, on a movement of a finger of hand 201on hand 205.

Control program 122 receives sensor data associated with a mapped hand(508) such as hand 205. In various embodiments, control program 122receives sensor data as a set of sensor data from sensors 124 associatedwith one or more motions of hand 201. A set of sensor data, which, maybeused as a set of user inputs, received from sensors 124 or from sensors150 by control program 122 is a set of one or more sensor data or a setof one or more user inputs. For example, control program 122 may receivesensor data from sensors 124 or sensors 150 indicating a touch of adigit on hand 201 on hand 205, a tapping of a digit of hand 201 on hand205, a slide of a digit of hand 201 on hand 205, a touch with a slide onone or more surfaces of hand 205, touches by multiple digits of hand 201on hand 205, a slide of a digit of hand 201 a curled of a palm of hand205, or the like. In an embodiment, control program 122 receives sensordata from sensors associated with hand 205. In one embodiment, controlprogram 122 receives sensor data from sensors associated with or on bothhand 201 and hand 205. In one example, control program 122 receivessensor data from sensors 124 indicating a tap on a location on the palmof hand 205 by a finger of hand 201. The location tapped, for example,corresponds to a content on transparent display 140 (e.g., determinedbased, at least in part, on mapping of hand 205 in steps 504 and 506).In various embodiments, control program 122 initiates by a proximity totransparent display 140 (e.g., to analyze received sensor data todetermine actions or instructions related to or for transparent display140). For example, control program 122 may be configured to start ananalysis of sensor data when computer 120 is within a pre-set distance(e.g., ten feet) of transparent display 140 (e.g., determined usingknown location determination methods).

Control program 122 determines whether the sensor data is associatedwith a content (decision 510). In various embodiments, control program122 includes the code and program routines to correlate sensor dataassociated with a motion of hand 201 on hand 205 with various actions oftransparent display 140 such as unlock transparent display. For example,control program 122 may be configured to correlate running a finger ofhand 201 over palm 205A of cupped hand 205 to unlocking transparentdisplay. In various embodiments, control program 122 also includesinstructions associating sensor data or a set of sensor data totransparent display 140 selecting and/or manipulating content on one ormore sides of transparent display 140.

In response to determining that the sensor data is not associated with acontent (no branch, decision 510), control program 122 sendsinstructions to perform action associated with sensor data (511) totransparent display 140. In various embodiments, control program 122identifies sensor data associated with a movement of a digit of hand 201with respect to hand 205 corresponding to an action such as unlocktransparent display 140, lock transparent display 140, or to power offtransparent display 140. For example, control program 122 receivessensor data from sensors 124 a cupped palm of hand 205 as a finger ofhand 201 transverses the cupped palm of hand 205 indicating that anaction (e.g., unlock transparent display 140), and response, controlprogram 122 sends instructions to transparent display 140 to unlock.

In another example, control program 122 sends instructions to locktransparent display 140 in response to sensor data or user input fromsensors 124 indicating a finger of hand 201 circles the outside of palm201A. In yet another example, control program 122 receives sensor datafrom sensors 124 determining that a digit of hand 201 creates an Xacross a side of hand 205, and in response, sends instructions to poweroff transparent display 140. In an embodiment, control program 122includes instructions for one or more actions associated with sensordata from sensors 150. For example, control program 122 determines thata first of hand 201 detected by a piezoelectric film as sensor 150 onhand 201 and sends an instruction to transparent display 140 to lock.

In some embodiments, control program 122 receives a user input from UI125 indicating an action of transparent display 140. For example,control program 122 receives a user input from UI 125 (e.g., a two-sidedremote input device in computer 120) when a user presses a large buttonon a two-sided smart remote input device configured to unlocktransparent display 140 and in response, control program 122 sendsinstructions to transparent display 140 to unlock. Similarly, whencontrol program 122 receives a user selection of a power off icon on atwo-sided touch screen for UI 125, then control program 122 sendsinstructions to transparent display 140 to power off. Control program122 is not intended to be limited to the examples of received sensordata and UI 125 user inputs in the examples discussed above but mayinclude other examples of other sensor data or other UI 125 inputsconfigured in control program 122 to actions of transparent display 140not associated with a content on transparent display 140.

Control program 122 determines that the sensor data is associated withcontent (yes branch, decision 510). In various embodiments, controlprogram 122 determines received sensor data (e.g., from either sensors124 or sensors 150) is associated with a selection of one or morecontent or a manipulation of one or more content (e.g., moving,deleting, flipping, etc. on transparent display 140).

In various embodiments, control program 122 determines a selection ofone or more content by a touch or a tap on a location of hand 205 by afinger of hand 201 mapped to one or more images, graphs, words, or othercontent on transparent display 140. For example, control program 122receives sensor data indicating a tap of a finger on hand 201 on alocation on palm 205A of hand 205 and determines a selection of thecontent on a side transparent display 140 corresponding to the locationon hand 205 tapped by a finger on hand 201 (e.g., based on mapping ofpalm 205A of hand 205 to side 210A of transparent display 140 in step504). In another example, control program 122 receives sensor dataindicating that a finger on hand 201 circles an area on the palm of hand205. In response, control program 122 determines that content ontransparent display 140 corresponding to the area circled by the fingerof hand 201 is selected (e.g., the selected content corresponds the areainside the circle on hand 205 by the finger of hand 201 based on mappingof palm 205A).

In some embodiments, control program 122 receives a user input on UI 125indicating a selection of content on transparent display 140. Forexample, control program 122 receives a user from UI 125 as either atwo-sided remote input device (e.g., included in computer 120 orindependent of computer 120) or two-sided touch screen included incomputer 120 indicating a selection or a selection of content and anaction associated with selected content. In an example, a user selects asmall button on UI 125 as a two-sided remote input device mapped to acontent on one side of transparent display 140. The press of a smallbutton in a grid of buttons on a side of UI 125 as a two-sided remoteinput device identifies a selection of the mapped content on transparentdisplay 140. In some embodiments, control program 122 receives userinput from both sides of UI 125 (e.g., either from buttons associatedwith content on both sides of the two-sided remote device for UI 125 ora touch on both sides of a two-sided touch screen for UI 125).Similarly, control program 122 receives a user input such as a touch ona two-sided touch screen in computer 120 indicating a selection ofcontent one transparent display 140 corresponding to the touchedlocation on UI 125. In this example, the touch on UI 125 is in locationthat is not an icon for an action or symbol for a pull-down menu used toa user select and communicate actions (e.g., lock display, power off,etc.) to control program 122.

Control program 122 determines whether the sensor data is associatedwith more than one side of transparent display 140 (decision 512).Responsive to determining that received sensor data that is notassociated with more than one side of transparent display 140 (nobranch, decision 512), control program 122 sends instructions tomanipulate and display content (514) to transparent display 140. Inresponse to received sensor data from sensors 124 or sensors 150associated with a gesture or motion of hand 201 on hand 205, controlprogram 122 determines instructions to send to transparent display 140to manipulate and display content. The instructions in control program122 associated with sensor data may be to manipulate content (e.g., tomove, delete, flip, etc.) and display content. In various embodiments,control program 122 determines how to manifest content or manipulatecontent on transparent display 140 in response to various receivedsensor data from sensors 124 and/or sensors 150. For example, aspreviously discussed, control program 122 based, at least in part, onreceived sensor data, determines a touch on a location on palm 205A ofhand 205 by a digit of hand 201 includes a slide to a second location.In response, control program 122 sends instructions to transparentdisplay 140 to move selected content to the second location (e.g.,determined by sensor data and mapping of hand 205 to transparentdisplay) and display selected content on the second location oftransparent display 140 (e.g., as depicted in FIGS. 2A and 2B).

In another example, control program 122 sends instructions totransparent display 140 to delete a selected content (e.g., a graph)when sensor data from sensors 124 indicate three taps on a location hand205 by a finger of hand 201. For example, a finger of hand 201 tapsthree times in the center of backside 205B of hand 205 and controlprogram 122 sends instructions to erase the content (e.g., the graph) inthe center of side 210B on transparent display 140.

In yet another example, control program 122 receives data from sensors124 indicating a double tap in the center of the palm of hand 205 andaccordingly, control program 122 sends instructions to transparentdisplay 140 to transfer the content (e.g., a picture) corresponding tothe double tapped location to the center of the front side 210A oftransparent display 140.

Control program 122 is not limited to the manipulation of content orinstructions corresponding to received sensor data as discussed in theexamples above but, may include a number of other actions ormanipulation of content on transparent display 140 in response to othersensor data received from sensors 124 and/or sensors 150 associated withmovements of hand 201 on hand 205.

In some embodiments, control program 122 receives user input from UI 125as either a two-sided remote input device or a two-sided touch screen incomputer 120 indicating a manipulation and display of content on oneside of transparent display 140. For example, control program 122receives a user input such as a depression of a small button associatedwith content on a location of transparent display 140 and a selection ofa large button associated with deleting content and in response, sendsinstructions to transparent display 140 to delete the selected content.

Control program 122 monitors sensors 124 and sensors 150 for additionalsensor data (516). If additional sensor data is received by controlprogram 122, the program returns to step 508. In an embodiment, controlprogram 122 receives sensor data indicating an end of control program122. For example, control program 122 receives sensor data indicatinghand 201 swipes hand 201 up along the length of hand 205 and controlprogram 122 ends.

In various embodiments, control program 122 receives sensor data that isassociated with more than one side of transparent display 140 (yesbranch, decision 512). For example, control program 122 may receivesensor data from one or more of sensors 124 or sensors 150 associatedwith two-sides of hand 205 indicating a selection of content and/or amanipulation of content associated with more than one side oftransparent display 140.

For example, control program 122 in response to receiving sensor dataindicating a double tap on a location on one side of hand 205 determinesthat the selected content should be flipped to the other side oftransparent display 140 (e.g., the sensor data is associated with morethan one side of transparent display 140). In another example, controlprogram 122 may receive sensor data indicating touches on multiple sidesof hand 205. As depicted in FIGS. 4A and 4B, sensor data indicates atouch by a finger on backside 205B of hand 205 and a touch on palm 205Aassociated with content on two sides of transparent display 140.

In an embodiment, control program 122 determines user inputs on UI 125are associated with more than one side of UI 125 (e.g., when UI 125 is atwo-sided remote input device or a two-sided touch screen). For example,when control program 122 receives in indication of a user pressing smallbutton on each side of a two-sided smart remote device as UI 125 whenthe buttons are associated with a location on transparent display 140then, control program 122 determines user inputs are associated withboth sides of transparent display 140. Similarly, when control program122 receives an indication of a user touch on each side of a two-sidedtouch screen (e.g., not on an icon for an action), then control program122 determines the user input is associated with two sides oftransparent display 140.

Responsive to determining sensor data is associated with more than oneside of the hand 205; control program 122 determines content selectedfor each side of transparent display (522). In various embodiments,based, at least in part, on received sensor data, control program 122determines if content on transparent display 140 is selected on one ormore sides of transparent display 140. For example, computer 120 usingcontrol program 122 receives sensor data indicating the selection of asingle content on one side of transparent display moving to a secondside of transparent display 140. For example, control program 122receives sensor data indicating a touch on a location on one side ofhand 205 by a finger and a slide to a second location on the oppositeside of hand 201 (e.g., as depicted in FIGS. 3A and 3B).

In another example, as depicted in FIGS. 4A and 4B, control program 122receives sensor data associated with more than one content on more thanone side of transparent display 140. In response to received sensor datafrom sensors in computer 120C (e.g., a smart watch or smart armband)indicating muscle movements associated with two digits of hand 201 onboth palm 205A and backside 205B of hand 205, computer 120 using controlprogram 122 (not depicted) determines that content is selected on bothsides of transparent display 140. Based on sensor data indicatingpointer finger of hand 201 is on a location of backside 205B of hand 205corresponding to a mapped to content 411 (e.g., a picture of a flower)and sensor data indicating a thumb of hand 201 is on a location of palm205A of hand 205 corresponding to mapped to content 412 (e.g., data ornumbers) on side 210A of transparent display 140, control program 122determines a content is selected on both sides of transparent display140.

Control program 122 determines whether content is moving from one sideof transparent display 140 to the other side of transparent display 140(decision 524). Control program 122 determines that the content selectedfor each side of transparent display 140 is not moving from one side oftransparent display 140 to the other side (no branch, decision 524) oftransparent display 140 based, at least in part, on received sensordata. For example, as depicted in FIGS. 4A and 4B discussed above,control program 122 in computer 120C determines from received sensordata associated with the finger of hand 201 on palm 205A and the thumbof hand 201 on backside 205B of hand 205 corresponding to content 411(e.g., a picture of a flower) on side 210B and content 412 (e.g.,numbers) on side 21A of transparent display 140. Additionally, sensordata received by computer 120 indicates the finger of hand 201 slidesdownward to a second location on backside 205B of hand 205. At the sametime, computer 120 receives sensor data indicating the thumb of hand 201is sliding upward to a second location on palm 205A of hand 205.Computer 120 (e.g., using control program 122 not depicted) determinesthat neither content 411 nor content 412 is moving to another side oftransparent display 140.

In one embodiment, control program 122 receives a user input on UI 125where UI 125 is a two-sided touch screen for computer 120 or a two-sidedremote input device indicating that content is manipulated on one sideof transparent display 140. For example, control program 122 receives auser input associated with selection of a button associated with alocation on each side of UI 125 and a larger button on the top of oneside of the two-sided remote input device as UI 125 associated with aselection to move content. Control program 122 receives a user inputindicating a target location by selection of a small button (e.g.,mapped to a location) on the same side of two-sided remote input deviceUI 125 as the pressed larger button to move content.

Responsive to determining that content is not moving from one side oftransparent display 140 to the other side of transparent display 140,control program 122 sends instructions to manipulate and display content(528) to transparent display 140. For example, as previously discussedwith respect to FIGS. 4A and 4B, responsive to receiving sensor dataindicating a downward movement of the finger of hand 201 on backside205B, control program 122 determines a movement or transfer of content411 (e.g., flower) downward to the mapped location on transparentdisplay 140 corresponding to the second location of the finger aftersliding on backside 205B. In the above example, simultaneously, inresponse to received sensor data, control program 122 determines asecond location for content 412 (e.g., numbers) on side 210A oftransparent display 140 corresponding to mapping of the second locationof the thumb of hand 201 after sliding up on palm 205A of hand 205. Inresponse to determining second locations for content 411 and content412, control program 122 sends instructions to transparent display 140to perform movements of content 411 and content 412 and then, to displaymoved content (e.g., content 411 and content 412 in second locations).

In some embodiments, control program 122 receives one or more inputs onUI 125 as a two-sided remote input device or a two-sided touch screen tomove one or more content from one location on a side of transparentdisplay 140 to another location on the same side of transparent display140. In response, control program 122 sends instructions to transparentdisplay 140 to move selected content to a second location on the sameside of transparent display 140 for display.

Control program 122 monitors sensors 124 and sensors 150 for additionalsensor data (530). If additional sensor data is received by controlprogram 122, the program returns to step 508. In an embodiment, controlprogram 122 receives sensor data from a user indicating the end of theprogram, for example, if received sensor data indicates hand 201 swipeshand 201 up along the length of hand 205, control program 122 ends. Inan embodiment, control program 122 receives a user input on UI 125 as aselection of a button or an icon indicating a program end.

Control program 122 determines that content is moving from one side oftransparent display 140 to the other side of transparent display 140(yes branch, decision 524). In various embodiments, control program 122determines that content is moving from one side of transparent display140, based, at least in part, on received sensor data. In an embodiment,control program 122 determines that content is moving from one side oftransparent display 140 to the other side of transparent display 140based on a user input on a two-sided remote electronic device as UI 125or on a two-sided touch screen for UI 125 in computer 120.

In one embodiment, control program 122 receives sensor data indicatingthat a digit of hand 201 slides around a side of the palm or edge ofhand 205 to a second location on an opposite side of hand 205. Forexample, as depicted in FIGS. 3A and 3B, a finger on hand 201 taps ortouches the palm of hand 205 and then, slides around the side of thepalm of hand 205 to the backside of hand 205. In response, controlprogram 122 receives sensor data from sensors 124 in computer 120 (e.g.,a smart ring) associated with the first location on the palm mapped to alocation and content on front side 210A. In this example, controlprogram 122 also receives sensor data associated with the slide to asecond location on the backside of hand 205 mapped to a second locationon side 210B of transparent display 140.

In an embodiment, control program 122 receives sensor data from sensors124 or 150 configured to flip a selected content from one side oftransparent display 140 to the other or a second side of transparentdisplay 140. For example, control program 122 receives sensor data fromsensors 124 or sensors 150 indicating to control program 122 that thecontent displayed on each side of transparent display 140 is flipped tothe other side of transparent display 140. For example, when controlprogram 122 receives sensor data indicating hand 205 has flipped over180 degrees while a finger on hand 201 approximately orthogonal to thepalm of hand 205 also flips 180 degrees (e.g., from pointing up topointing down on hand 205), control program 122 determines an action,such as flipping content displayed on front side of transparent display140 to the backside of transparent display 140 and vice versa (e.g.,backside content flipped to front side). Flipping sides of transparentdisplay 140, for example, includes flipping or moving images or contentdisplayed on the backside 210B of transparent display 140 to front side210A of transparent display 140. Control program 122 is not limited todetermining a movement or manipulation of content according to theexamples discussed above but may receive other sensor data correlated toother actions or manipulation of content on transparent display 140.

In one embodiment, control program 122 receives input from UI 125 (e.g.,input to UI 125 as either a two-sided touch screen or a two-sided remoteelectronic device included in or with computer 120) indicating a flip ora move of content from one side of transparent display 140 to the otherside of transparent display 140. For example, UI 125 as a two-sidedremote input device receives a user selection on a small buttonassociated with a location of content on transparent display 140 and aselection of a larger button indicating a flip or a transfer of theselected content to a corresponding location on the other side oftransparent display 140. A similar method of selecting a content and anicon or a selection in a drop-down menu indicating a transfer of thecontent to a same or similar location on the other side of transparentdisplay 140 can be done with UI 125 as a two-sided touch screen.

In various embodiments, control program 122 in response to receivingsensor data indicating a transfer of content from a front side 210A toside 210B or vice versa on transparent display 140, control program 122determines an orientation of content on the second side of transparentdisplay 140 (526). In various embodiments, control program 122 may needto flip or reverse the orientation of content such as words, symbols,graphs, and the like for the content to be properly displayed forconsumption or reading from the other side of transparent display 140.Control program 122 determines the orientation of content on the secondside of transparent display 140.

For example, control program 122 determines an orientation of contenttransferring to a second side of transparent display 140 providing auser viewing the second side of transparent display 140 the same view ororientation of the content as a user viewing the content from the firstside of transparent display 140. For example, as depicted in FIG. 3B,control program 122 sends instructions to transparent display 140 toorient content 303 as content 303 is moved from side 210A to side 210Bof transparent display 140. In FIG. 3B, content 303 is depicted on side210A (top) and content 303 is depicted on side 210B (bottom) as a userstanding on side 210A would see content 303 after orienting the content(e.g., content on side 210B would appear backwards or reversed to theuser viewing from the front side 210A of transparent display 140).

After orienting content, control program 122 sends instructions tomanipulate and display content (528) to transparent display 140 asdiscussed above. For example, when control program 122 receives sensordata associated with a double tap of a location on one side of palm 205Amapped to a content such as a graph (not depicted) then, control program122 sends instructions for the selected content (e.g., the graph) on aside of transparent display 140 (e.g., on front side 210A) is to beflipped or transferred to the other side (e.g., back side 210B) oftransparent display 140 (e.g., to a corresponding location on theopposite side of transparent display 140) and displayed. In anotherexample, control program 122 sends instructions to flip and display thecontent on each side of transparent display 140 to the other side oftransparent display 140 responsive to receiving a selection of a buttonor an icon on UI 125 configured to reverse or flip content displayed oneach side of transparent display 140 (e.g., content on front side 210Aof transparent display 140 transfers to backside 210B of transparentdisplay 140).

In various embodiments, control program 122 monitors sensors foradditional sensor data (530). If no additional sensor data is received,control program 122 ends unless additional sensor data is received, inwhich case, control program 122 returns to step 508. In someembodiments, control program 122 monitors UI 125 as a two-sided remoteinput device or a two-sided touch screen for a user input associatedwith transparent display 140.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s).

FIG. 6 is block diagram 600 depicting components of a computer system inaccordance with at least one embodiment of the present invention. Asdepicted, FIG. 6 depicts the components of a computer system, which isan example of a system such as computer 120 within distributed dataprocessing environment 100, in accordance with an embodiment of thepresent invention. It should be appreciated that FIG. 6 provides only anillustration of one implementation and does not imply any limitationswith regard to the environments in which different embodiments can beimplemented. Many modifications to the depicted environment can be made.

Computer 120 can include processor(s) 604, cache 614, memory 606,persistent storage 608, communications unit 610, input/output (I/O)interface(s) 612 and communications fabric 602. Communications fabric602 provides communications between cache 614, memory 606, persistentstorage 608, communications unit 610 and input/output (I/O) interface(s)612.

Communications fabric 602 can be implemented with any architecturedesigned for passing data and/or control information between processors(such as microprocessors, communications and network processors, etc.),system memory, peripheral devices and any other hardware componentswithin a system. For example, communications fabric 602 can beimplemented with one or more buses.

Memory 606 and persistent storage 608 are computer readable storagemedia. In this embodiment, memory 606 includes random access memory(RAM). In general, memory 606 can include any suitable volatile ornon-volatile computer readable storage media. Cache 614 is a fast memorythat enhances the performance of processor(s) 604 by holding recentlyaccessed data and near recently accessed data, from memory 606.

Program instructions and data used to practice embodiments of thepresent invention are stored in persistent storage 608 for executionand/or access by one or more of the respective processor(s) 604 viacache 614. In this embodiment, persistent storage 608 includes amagnetic hard disk drive. Alternatively, or in addition to a magnetichard disk drive, persistent storage 608 can include a solid-state harddrive, a semiconductor storage device, a read-only memory (ROM), anerasable programmable read-only memory (EPROM), a flash memory or anyother computer readable storage media that is capable of storing programinstructions or digital information.

The media used by persistent storage 608 may also be removable. Forexample, a removable hard drive may be used for persistent storage 608.Other examples include optical and magnetic disks, thumb drives andsmart cards that are inserted into a drive for transfer onto anothercomputer readable storage medium that is part of persistent storage 608.

Communications unit 610, in these examples, provides for communicationswith other data processing systems or devices, including resources ofcomputer 120, sensors 150, and transparent display 140 and othercomputing devices not shown in FIG. 1. In these examples, communicationsunit 610 includes one or more network interface cards. Communicationsunit 610 may provide communications with either or both physical andwireless communications links. Program instructions and data used topractice embodiments of the present invention may be downloaded topersistent storage 608 through communications unit 610.

I/O interface(s) 612 allows for input and output of data with otherdevices that may be connected to computer 120, sensors 150, ortransparent display 140. For example, I/O interface(s) 612 may provide aconnection to external device(s) 616 such as a keyboard, a keypad, atouch screen, a microphone, a digital camera and/or some other suitableinput device. External device(s) 616 can also include portable computerreadable storage media, for example, devices such as thumb drives,portable optical or magnetic disks and memory cards. Software and dataused to practice embodiments of the present invention can be stored onsuch portable computer readable storage media and can be loaded ontopersistent storage 608 via I/O interface(s) 612. I/O interface(s) 612also connect to a display 618.

Display 618 provides a mechanism to display data to a user and may be,for example, a computer monitor. Display 618 can also function as atwo-sided touch screen, such as a display of a tablet computer or atransparent display 140.

The programs described herein are identified based upon the applicationfor which they are implemented in a specific embodiment of theinvention. However, it should be appreciated that any particular programnomenclature herein is used merely for convenience, and thus theinvention should not be limited to use solely in any specificapplication identified and/or implied by such nomenclature.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be any tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers, and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general-purpose computer, a special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, a segment, or aportion of instructions, which comprises one or more executableinstructions for implementing the specified logical function(s). In somealternative implementations, the functions noted in the blocks may occurout of the order noted in the Figures. For example, two blocks shown insuccession may, in fact, be executed substantially concurrently, or theblocks may sometimes be executed in the reverse order, depending uponthe functionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the invention.The terminology used herein was chosen to best explain the principles ofthe embodiment, the practical application, or technical improvement overtechnologies found in the marketplace, or to enable others of ordinaryskill in the art to understand the embodiments disclosed herein.

What is claimed is:
 1. A method comprising: receiving, by one or morecomputer processors, a first set of mapping sensor data associated witha plurality of systematic movements of the second hand of the user on afirst side of the first hand of the user; mapping, by one or morecomputer processors, the first set of mapping sensor data to the firstside of the first hand of the user; mapping, by one or more computerprocessors, the first side of the first hand of the user to the firstside of the transparent display; receiving, by one or more computerprocessors, a second set of mapping sensor data associated with aplurality of movements of the second hand of the user on a second sideof the first hand of the user; mapping, by one or more computerprocessors, the second set of mapping sensor data to the second side ofthe first hand of the user; mapping, by one or more computer processors,the second side of the first hand of the user to the second side of thetransparent display; receiving, by one or more computer processors, afirst set of sensor data associated with one or more movements of thesecond hand of user associated to the first hand of the user, whereinthe one or more sets of mapping sensor data and the first set of sensordata are determined based on muscle movements or vibrations detectedfrom the plurality of systematic movements of the second hand of theuser on the first hand of the user and the one or more movements of thesecond hand of the user on the first hand of the user, and wherein theone or more sets of sensor data and the first set of sensor data arereceived from one or more sensors in at least one of: a smart ring, asmart watch, a fitness tracking device, a smart armband, or a sensorglove with processing capability associated with the second hand;determining, by one or more computer processors, the first set of sensordata corresponds to at least one content on the transparent display;determining, by one or more computer processors, whether the first setof sensor data is associated with more than one side of the transparentdisplay; responsive to determining the first set of sensor data is notassociated with more than one side of the transparent display,determining, by one or more computer processors, a manipulation of theat least one content on a side of the transparent display correspondingto the first set of sensor data; sending, by one or more computerprocessors, one or more instructions to the transparent display todisplay the at least one content based, at least in part, on thedetermined manipulation of the at least one content and the mapping ofthe transparent display; responsive to determining the first set ofsensor data is associated with more than one side of the transparentdisplay, determining, by one or more computer processors, that the atleast one content is moving from one side of the transparent display toa second side of the transparent display based, at least in part, on thefirst set of sensor data; determining, by one or more computerprocessors, an orientation of the at least one content on the secondside of the transparent display; and sending, by one or more computerprocessors, one or more instructions to the transparent display todisplay the at least one content on the second side of the transparentdisplay based, at least in part, on the first set of sensor data.